Ideally spider should throw an error when local and remote tables have
conflict definitions.

This fixes the test for --view-protocol and
--mysqld=--loose-spider-disable-group-by-handler

…ider_db_result in spider_db_store_result

This prevents segv on NULL result_list->current->result on handler
calls that are not the first. Reasons supporting this change:

- spider_db_result::limit_mode is not called anywhere else. The only
  calls to limit_mode method are on spider_db_conn
- it is very unlikely (impossible?) for the connection to change from
  one backend to another between execution of sql statements and
  storing result:

/* in spider_bg_conn_action: */
                if (dbton_handler->execute_sql(
                  sql_type,
                  conn,
                  result_list->quick_mode,
                  &spider->need_mons[conn->link_idx])
// [... 9 lines elided]
                    if (!(result_list->bgs_error =
                      spider_db_store_result(spider, conn->link_idx,
                        result_list->table)))

  this also means it is very unlikely (impossible?) for the backend
  type (dbton_id) to differ between conn->db_conn and
  result_list->current->result, also considering that
  spider_db_result::dbton_id comes from spider_db_conn::dbton_id:

spider_db_result::spider_db_result(
  SPIDER_DB_CONN *in_db_conn
) : db_conn(in_db_conn), dbton_id(in_db_conn->dbton_id)

Since this was the only call to spider_db_result::limit_mode, we also
remove the method altogether.

…pider tests

This also has the side benefits of catching --view-protocol failures
in CI, as --view-protocol often causes spider gbh not to be created

Any InnoDB write workload is marking data pages in the buffer pool dirty.
To make the changes durable, it suffices to write to the write-ahead log
to facilitate crash recovery. The longer we keep pages dirty in the
buffer pool, the better, because the same pages could be modified again
and a single write to the file system could cover several changes.

Eventually, we must write out dirty pages, so that we can advance the
log checkpoint, that is, allow the start of the recovery log to be
discarded. (On a clean shutdown, a full checkpoint to the end of the log
will be made.)

A write workload can be bound by innodb_buffer_pool_size
(we must write out changes and evict data pages to make room for others)
or by innodb_log_file_size
(we must advance the log checkpoint before the tail of the circular
ib_logfile0 would overwrite the previous checkpoint).

In innodb_log_file_size bound workloads, we failed to set an optimal
target for the next checkpoint LSN. If we write out too few pages, then
all writer threads may occasionally be blocked in log_free_check() while
the buf_flush_page_cleaner() thread is resolving the situation. If we
write out too many pages, then the I/O subsystem will be loaded
unnecessarily and there will be some write amplification in case some of
the unnecessarily written pages would be modified soon afterwards.

log_close(): Return the target LSN for buf_flush_ahead(lsn),
bitwise-ORed with the "furious" flag, or the special values 0
indicating that no flushing is needed, which is the usual case.

log_checkpoint_margin(): Use a similar page checkpoint target as
log_close() for the !not_furious case.

mtr_flush_ahead(): A wrapper for buf_flush_ahead().

mtr_t::commit_log_release(): Make some common code non-inline in order
to reduce code duplication.

buf_flush_ahead(lsn, furious=false): Avoid an unnecessary wake-up of the
page cleaner if it is scheduled to wake up once per second.

Co-developed-by: Alessandro Vetere
Reviewed by: Vladislav Lesin

btr_search_lazy_free(): Check for !table->id. It is possible that
dict_sys_t::remove(table, ...) has started executing but has not
reached the final part yet. If that is the case, let
dict_sys_t::remove() observe that table->freed_indexes is empty
and let it free the table object.

Reviewed by: Thirunarayanan Balathandayuthapani
Tested by: Saahil Alam

The debug parameter innodb_trx_rseg_n_slots_debug is not compatible
with the bug fix that was implemented in MDEV-30671 and last revised
in MDEV-33213, which leads to bogus assertion failures in
trx_assign_rseg_low(), regarding look_for_rollover. When needed,
this parameter can be better simulated by adding a DBUG_EXECUTE_IF to
trx_undo_seg_create() or one of its callers.

…overflow

Problem:
========
- When an R-tree root page becomes full and requires splitting,
InnoDB follows a specific root-raising procedure to maintain
tree integrity. The process involves allocating a new page
(Page X) to hold the current root's content, preserving the
original root page number as the tree's entry point, and
migrating all existing records to Page X.

The root page is then cleared and reconstructed as an
internal node containing a single node pointer with an
MBR that encompasses all spatial objects on Page X.
Subsequently, InnoDB should split the records on Page X
into two spatially optimized groups using the
pick_seeds() and pick_next() algorithms,
creating a second page (Page Y) for Group B records
while retaining Group A records on Page X.

After records are redistributed between Page X and Page Y,
the recalculated MBR for Page X must remain within
or be smaller than the original MBR stored in the
root page's node pointer.

Bug scenario:
============
- When root page 4 becomes full, it triggers a split operation
where the content is copied to page 7 and root page 4 is cleared
to become an internal node.
- During the first split attempt on page 7, Group 1 overflows
and remaining entries are reassigned to Group 2.
- A new page 8 is created and the remaining entry record
is inserted, but the combined size of the remaining entry
record and new record exceeds the page size limit.
- This triggers a second split operation on page 7, where
Group 2 overflows again and entries are moved back to Group 1.
- When the new record is finally inserted into page 7,
it causes the MBR (Minimum Bounding Rectangle) for page 7
to expand beyond its original boundaries.
- Subsequently, when InnoDB attempts to update the parent
page 4 with the new MBR information, it fails to locate
the corresponding internal node, leading to spatial
index corruption and the reported failure.

Problem:
========
- Second split operation should happen on page 8, not on page 7.
- split_rtree_node() considers key_size to estimate
record sizes during the splitting algorithm, which fails to
account for variable-length fields in spatial records.
- In rtr_page_split_and_insert(), when reorganization
succeeds, InnoDB doesn't attempt the insert the entry

Solution:
========
rtr_page_split_and_insert(): InnoDB should do insert the
tuple when btr_page_reorganize() is successful.

rtr_page_split_and_insert(): Use the overflow page
for consecutive split operation.

split_rtree_node(): Store the record length for each
record in r-tree node. This should give proper
estimation while determining the group entries and
also helpful in overflow validation

rtree_mbr_from_wkb::n_node: Change the data type from int to uint16_t
and pack it together with the added field key_len. In this way, an
increase of sizeof(rtree_mbr_from_wkb) will be avoided.

row_raw_format_str(): Treat the invalid value charset_coll==0 as binary.
This could be invoked on FTS_%_CONFIG.key or SYS_FOREIGN.ID
or possible other key columns.

dtype_is_utf8(): Merge to its only caller row_raw_format_str().

row_raw_format(): Add debug assertions and comments to document
when dtype_get_charset_coll(prtype) may be 0.

Fixed by marking MRG_MYISAM changes explicitly as a DDL

The "FETCH FIRST n ROWS WITH TIES" was not enforced when the SELECT used
"using index for group-by".

This was caused by an optimization which removed the ORDER BY clause
when the GROUP BY clause prescribed a compatible ordering.

Other GROUP BY strategies used workarounds to still handle WITH TIES,
see comment to using_with_ties_and_group_min_max() in this patch for
details. QUICK_GROUP_MIN_MAX_SELECT didn't have a workaround.

Fix this by disabling removal of ORDER BY when
QUICK_GROUP_MIN_MAX_SELECT is used.

…EP_START_POSITION

MDEV-35904 - backport MDEV-19210 to 10.11 as referenced
by unset environment variables become warnings.

We used to run `systemctl set-environment` to pass
_WSREP_START_POSITION. This is bad because:

* it clutter systemd's environment (yes, pid 1)
* it requires root privileges
* options (like LimitNOFILE=) are not applied

Let's just create an environment file in ExecStartPre=, that is read
before ExecStart= kicks in. We have _WSREP_START_POSITION around for the
main process without any downsides.

…nd collation associated with views, etc

In Query_log_event::do_apply_event there is comparison is used
character set in event same as cached charset and if not used
charset is changed. Unfortunately, it was done only if thread is
replica thread.

Fixed by adding condition for Galera applier thread so that
comparison is done leading to charset change if event
had different charset.

The Debian library dependencies are auto detected and populated
as part of shlibs:Depends and the explicit population
is suspectable to distro packaging changes.

Also fix the server gssapi plugin to the compiled server
dependency consistent with other server plugins.

Problem was that thd->lex->m_sql_cmd is not always set
especially when user has not provided ENGINE=xxx so
requesting option_storage_engine_name from there is not
safe.

Fixed by accessing thd->lex->m_sql_cmd only when user
has used ENGINE= and if not using ha_default_handlerton
and requesting engine name after it.

check for thd->killed in the lexer

…table upon INSERT DELAYED

sequences, just as triggers or check constraints, disable DELAYED

Test case only. The bug is fixed by cherry-pick of 2d5db53
"Simplify NEXTVAL(sequence) when used with DEFAULT"

remove the fix for MDEV-25243. It's no longer needed, because
a routine can no longer be re-parsed in the middle of a statement.

…re than 100 ORs

(Variant 2)
SEL_TREE* tree_or(SEL_TREE *X, SEL_TREE *Y) tries to conserve memory by
reusing object *X for the return value when possible.

MDEV-34620 has added logic to disable construction of index_merge plans
for N-way ORs with large N. That logic interfered with object reuse logic:

for the parameters of:
X = SEL_TREE{ trees=[key1_treeA, key2_treeB]}
Y = SEL_TREE{ trees=[key1_treeC]}

we would decide to reuse object X.
For key1, we would produce key_or(key1_treeA, key1_treeC)
but key2_treeB would be just left there.
Then, we would construct a range scan from key2_treeB.

Fixed by moving the "disable building index_merge plans" logic into a
location where it would not interfere with object reuse logic.

* remove the file to be --repeat friendly
* specify the correct defaults-group-suffix. Spider doesn't use the
  conventional .1 group, so MYSQLD_CMD skips a lot of config options,
  in particular it doesn't read --tmpdir and creates files in /tmp

MDEV-37726 moved wsrep-start-position to INSTALL_RUNDATADIR
and made the latter to be created by systemd-tmpfiles.

Now postin scriptlet has to run systemd-tmpfiles explicitly
to make sure INSTALL_RUNDATADIR exists before restarting
the server.

followup for 649216e

table->pos_in_table_list is problematic, it can become
dangled when an SP is reparsed. Let's avoid it and directly
prefer TABLEs where a sequence is ready for inserts.

followup for 516f68a

…lumns

(Preparation for the main patch)

set_statistics_for_table() incorrectly treated indexes with all NULL
values the same as indexes with no statistics, because avg_frequency
is 0 in both cases. This caused the optimizer to ignore valid EITS
data and fall back to engine statistics.

Additionally, KEY::actual_rec_per_key() would fall back to engine
statistics even when EITS was available, and used incorrect pointer
comparison (rec_per_key == 0 instead of nullptr).

Fix by adding Index_statistics::stats_were_read flag to track per-index
whether statistics were actually read from persistent tables, and
restructuring actual_rec_per_key() to prioritize EITS when available.

…olumns

When all values in an indexed column are NULL, EITS statistics show
avg_frequency == 0. This commit adds logic to distinguish between
"no statistics available" and "all values are NULL" scenarios.

For NULL-rejecting conditions (e.g., t1.col = t2.col), when statistics
confirm all indexed values are NULL, the optimizer can now return a
very low cardinality estimate (1.0) instead of unknown (0.0), since
NULL = NULL never matches.

For non-NULL-rejecting conditions (e.g., t1.col <=> t2.col),
normal cardinality estimation continues to apply since matches are possible.

Changes:
- Added KEY::rec_per_key_null_aware() to check nulls_ratio from column
  statistics when avg_frequency is 0
- Modified best_access_path() in sql_select.cc to use the new
  rec_per_key_null_aware() method for ref access cost estimation
- The optimization works with single-column and composite indexes,
  checking each key part's NULL-rejecting status via notnull_part bitmap

  - MDEV-34330: System package SYS.DBMS_UTILITY

Added the following functions:
- GET_TIME - Return a measure of current time in hundredths of a second
- FORMAT_ERROR_BACKTRACE - Display the call stack at the point where an
exception was raised, even if the procedure is called from an exception
handler in an outer scope
- FORMAT_ERROR_STACK - Format the current error stack

Resulting line numbers of desired SQL statement lines in package
routines are based on the line numbers of the "SHOW CREATE" command for
the package body.

Known limitation:
Outputs similar to those of Oracle, in requesting for the strings not just
from the stack's bottom function, but also those in the middle, are not
accommodated. Output is always, as if the request is coming from the stack's
bottom function.

 - MDEV-19635: System package SYS.DBMS_SQL

This patch adds support for dynamic SQL to parse any data manipulation
language (DML) or data definition language (DDL) statement using PL/SQL.

Added the following functions:
- OPEN_CURSOR - Open a new cursor
- EXECUTE - Execute a given cursor

Added the following procedures:
- PARSE - Parse the given statement in the given cursor
- CLOSE_CURSOR - Close the cursor when no longer needed for a session

As of the moment, '1', corresponding to Oracle's NATIVE DBMS_SQL constant,
as PARSE's procedure's (language_flag) 3rd argument, is the only one
accepted.